World Modelers PI meeting demo

Benjamin M. Gyori, Harvard Medical School, 7/29/2018

In [109]:
%matplotlib inline
import json
import numpy
import pandas
import matplotlib.pyplot as plt
from IPython.display import Image
plt.rcParams['font.size'] = 18
plt.rcParams['figure.figsize'] = [10, 8]
from collections import Counter
from pysb.simulator import ScipyOdeSimulator
def df_from_counter(c, idx): return pandas.DataFrame.from_dict(cnt, orient='index').reset_index().rename(columns={'index': idx, 0:'count'})
In [110]:
Image('demo/indra_concept_detailed.png')
Out[110]:
In [10]:
Image('demo/indra_apps.png')
Out[10]:

Import some INDRA modules

In [11]:
from indra.sources import eidos, hume, sofia, cwms
from indra.assemblers import GraphAssembler, CAGAssembler, PysbAssembler
from indra.statements import Influence, Concept

Simple text-to-model assembly

We first look at a simple example to the text to model assembly pipeline. The use case here assumes an expert who aims to rapidly prototype a model based on assumptions they describe in English language.

In [12]:
Image('demo/nl_modeling.jpg', retina=True)
Out[12]:
In [13]:
Image('demo/model_mouth.png', retina=True)
Out[13]:

Describe a model of food insecurity

In [111]:
text = """A significant increase in precipitation resulted in food insecurity and
a decrease in humanitarian interventions. Actually, food insecurity itself can
lead to conflict, and in turn, conflict can drive food insecurity.
Generally, humanitarian interventions reduce conflict."""

Read with Eidos via API, process into INDRA Statements

Eidos reading process

Courtesy: Becky Sharp, UA

In [15]:
Image('demo/eidos.png')
Out[15]:
In [112]:
eidos_processor = eidos.process_text(text, webservice='http://localhost:5000')
eidos_processor.statements
Out[112]:
[Influence(conflict(), food insecurity()),
 Influence(precipitation(positive), humanitarian interventions(negative)),
 Influence(precipitation(positive), food insecurity()),
 Influence(food insecurity(), conflict()),
 Influence(humanitarian interventions(), conflict(negative))]
In [17]:
Image('demo/influence.png')
Out[17]:

Assemble into graph and draw

In [113]:
ga = GraphAssembler(eidos_processor.statements)
ga.make_model()
ga.save_pdf('text_to_model.png')
Image('text_to_model.png', width=800)
Out[113]:

Assemble into PySB/ODE model and simulate

In [114]:
def assemble_pysb(stmts, reverse_effects=True):
    pa = PysbAssembler()
    pa.add_statements(stmts)
    model = pa.make_model(reverse_effects=reverse_effects)
    return model

def simulate_model(model, ts=None, pd=None):
    if not ts:
        ts = numpy.linspace(0, 1000, 100)
    sim = ScipyOdeSimulator(model, ts)
    if not pd:
        res = sim.run()
    else:
        res = sim.run(param_values=pd)
    df = res.dataframe
    df = df.rename(columns={'__s%d' % i: s.monomer_patterns[0].monomer.name
                            for i, s in enumerate(model.species)})
    return df

model = assemble_pysb(eidos_processor.statements, True)
df = simulate_model(model, ts=None, pd={'kf_f_deg_1': 1e-4})
df.plot()
Out[114]:
<matplotlib.axes._subplots.AxesSubplot at 0x18d569240>

Now let's see what happens in whe increase humanitarian intervention and decrease precipitation

Both conflict and food insecturity now decrease.

In [115]:
df = simulate_model(model, ts=None, pd={'humanitarian_interventions_0': 1e6, 'precipitation_0': 1e3,
                                        'kf_f_deg_1': 1e-3})
df.plot(ylim=[0, 15000])
Out[115]:
<matplotlib.axes._subplots.AxesSubplot at 0x18d41ceb8>

We can also modify the model simply by changing the text

In [116]:
text2 = text + ' Displacement causes food insecurity.'
In [118]:
df = simulate_model(assemble_pysb(eidos.process_text(text2, webservice='http://localhost:5000').statements),
                    ts=None, pd={'humanitarian_interventions_0': 1e6, 'precipitation_0': 1e3,
                                 'kf_f_deg_1': 1e-3})
df.plot(ylim=[0, 15000])
Out[118]:
<matplotlib.axes._subplots.AxesSubplot at 0x18cf12710>

Reading, assembly and modeling of food insecurity in South Sudan

In [119]:
Image('demo/eval_doc_text_model.png')
Out[119]:
In [34]:
Image('demo/demo_arch_pimtg.png')
Out[34]:
In [120]:
from run_eval import *
In [121]:
# Get the IDs of all the documents in the docs folder
docnames = sorted(['.'.join(os.path.basename(f).split('.')[:-1])
                   for f in glob.glob('docs/*.txt')],
                   key=lambda x: int(x.split('_')[0]))
exclude = '31_South_Sudan_2018_Humanitarian_Needs_Overview'
docnames = [d for d in docnames if d != exclude]
print('Using %d documents' % len(docnames))
Using 51 documents

On another note: indra.literature.elsevier_client and indra.literature.newsapi_client can be used to query for and collect reading corpora on demand (e.g. we collected Elsevier-published papers discussing "food security + intervention")

In [122]:
eidos_stmts = read_eidos(docnames)
cwms_stmts = read_cwms_sentences(extract_eidos_text(docnames), read=False)
hume_stmts = read_hume('bbn/wm_m6_0628.json-ld')
sofia_stmts = read_sofia('sofia/MITRE_June18_v1.xlsx')
6 stmts from 1_IPC_Alert_9_SouthSudan_Sept2017
93 stmts from 2_IFPRI
50 stmts from 3_ALPAN_Paper_10
4 stmts from 4_FAO
108 stmts from 5_annurev-publhealth-031816-044356
6 stmts from 6_FAO_news
0 stmts from 7_South_Sudan_KeyMessages_Sept2017
24 stmts from 8_EA_Seasonal Monitor_2017_08_11
26 stmts from 9_FAOSS Livestock Crisis Update 2015
151 stmts from 10_FAO_a-i5505e
114 stmts from 11_FAO_a-i5591e
8 stmts from 12_SSD21-February-2017
15 stmts from 13_SSD_8
13 stmts from 14_SSD09-May-2017
24 stmts from 15_FEWS NET South Sudan Famine Risk Alert_20170117
4 stmts from 16_South Sudan - Key Message Update_ Thu, 2018-01-25
25 stmts from 17_FS2011-3072
4 stmts from 18_FFP Fact Sheet_South Sudan_2018.01.17
2 stmts from 19_wfp284277
3 stmts from 20_wfp287168
98 stmts from 21_i8343en
20 stmts from 22_IPC_South_Sudan_AcuteFI_May2017_June-July2017
1 stmts from 23_ipc_south_sudan_key_messages_feb2017
29 stmts from 24_pnas.201409769
78 stmts from 25_2016_HNO_South Sudan
107 stmts from 26_South_Sudan_2018_Humanitarian_Needs_Overview
5 stmts from 27_South Sudan_ Floods - Sep 2017 _ ReliefWeb
74 stmts from 28_SOUTH SUDAN Food Security Outlook_102017_0
36 stmts from 29_SOUTH SUDAN_FSOU_122017_0
79 stmts from 30_South_Sudan_2017_Humanitarian_Needs_Overview
1 stmts from 32_sudantribune
8 stmts from 33_Achievement of household food security in South Sudan - Sudan Tribune_ Plural news and views on Sudan
3 stmts from 34_Floods displace hundreds in war-torn in South Sudan - Sudan Tribune_ Plural news and views on Sudan
7 stmts from 35_Food security practical plan for South Sudan - Sudan Tribune_ Plural news and views on Sudan
3 stmts from 36_Hunger forces 45,000 people out of Unity state homes - Sudan Tribune_ Plural news and views on Sudan
2 stmts from 37_Hunger kills more than 40 people in S
3 stmts from 38_Hunger looms in S
1 stmts from 39_Millions could face famine in South Sudan, warns WFP - Sudan Tribune_ Plural news and views on Sudan
6 stmts from 40_sudantribune
5 stmts from 41_sudantribune
77 stmts from 42_extreme-weather-resilience-global-food-system
5 stmts from 43_wfp280359
4 stmts from 44_wfp279459
4 stmts from 45_wfp278542
3 stmts from 46_wfp278541
3 stmts from 47_wfp287168
2 stmts from 48_wfp284277
29 stmts from 49_WFP-0000050282
52 stmts from 50_wfp280481
24 stmts from 51_wfp289178
11 stmts from 52_i8533en
1_IPC_Alert_9_SouthSudan_Sept2017
Reading docs/1_IPC_Alert_9_SouthSudan_Sept2017.txt
2_IFPRI
Reading docs/2_IFPRI.txt
3_ALPAN_Paper_10
Reading docs/3_ALPAN_Paper_10.txt
4_FAO
Reading docs/4_FAO.txt
5_annurev-publhealth-031816-044356
Reading docs/5_annurev-publhealth-031816-044356.txt
6_FAO_news
Reading docs/6_FAO_news.txt
7_South_Sudan_KeyMessages_Sept2017
Reading docs/7_South_Sudan_KeyMessages_Sept2017.txt
8_EA_Seasonal Monitor_2017_08_11
Reading docs/8_EA_Seasonal Monitor_2017_08_11.txt
9_FAOSS Livestock Crisis Update 2015
Reading docs/9_FAOSS Livestock Crisis Update 2015.txt
10_FAO_a-i5505e
Reading docs/10_FAO_a-i5505e.txt
11_FAO_a-i5591e
Reading docs/11_FAO_a-i5591e.txt
12_SSD21-February-2017
Reading docs/12_SSD21-February-2017.txt
13_SSD_8
Reading docs/13_SSD_8.txt
14_SSD09-May-2017
Reading docs/14_SSD09-May-2017.txt
15_FEWS NET South Sudan Famine Risk Alert_20170117
Reading docs/15_FEWS NET South Sudan Famine Risk Alert_20170117.txt
16_South Sudan - Key Message Update_ Thu, 2018-01-25
Reading docs/16_South Sudan - Key Message Update_ Thu, 2018-01-25.txt
17_FS2011-3072
Reading docs/17_FS2011-3072.txt
18_FFP Fact Sheet_South Sudan_2018.01.17
Reading docs/18_FFP Fact Sheet_South Sudan_2018.01.17.txt
19_wfp284277
Reading docs/19_wfp284277.txt
20_wfp287168
Reading docs/20_wfp287168.txt
21_i8343en
Reading docs/21_i8343en.txt
22_IPC_South_Sudan_AcuteFI_May2017_June-July2017
Reading docs/22_IPC_South_Sudan_AcuteFI_May2017_June-July2017.txt
23_ipc_south_sudan_key_messages_feb2017
Reading docs/23_ipc_south_sudan_key_messages_feb2017.txt
24_pnas.201409769
Reading docs/24_pnas.201409769.txt
25_2016_HNO_South Sudan
Reading docs/25_2016_HNO_South Sudan.txt
26_South_Sudan_2018_Humanitarian_Needs_Overview
Reading docs/26_South_Sudan_2018_Humanitarian_Needs_Overview.txt
27_South Sudan_ Floods - Sep 2017 _ ReliefWeb
Reading docs/27_South Sudan_ Floods - Sep 2017 _ ReliefWeb.txt
28_SOUTH SUDAN Food Security Outlook_102017_0
Reading docs/28_SOUTH SUDAN Food Security Outlook_102017_0.txt
29_SOUTH SUDAN_FSOU_122017_0
Reading docs/29_SOUTH SUDAN_FSOU_122017_0.txt
30_South_Sudan_2017_Humanitarian_Needs_Overview
Reading docs/30_South_Sudan_2017_Humanitarian_Needs_Overview.txt
32_sudantribune
Reading docs/32_sudantribune.txt
33_Achievement of household food security in South Sudan - Sudan Tribune_ Plural news and views on Sudan
Reading docs/33_Achievement of household food security in South Sudan - Sudan Tribune_ Plural news and views on Sudan.txt
34_Floods displace hundreds in war-torn in South Sudan - Sudan Tribune_ Plural news and views on Sudan
Reading docs/34_Floods displace hundreds in war-torn in South Sudan - Sudan Tribune_ Plural news and views on Sudan.txt
35_Food security practical plan for South Sudan - Sudan Tribune_ Plural news and views on Sudan
Reading docs/35_Food security practical plan for South Sudan - Sudan Tribune_ Plural news and views on Sudan.txt
36_Hunger forces 45,000 people out of Unity state homes - Sudan Tribune_ Plural news and views on Sudan
Reading docs/36_Hunger forces 45,000 people out of Unity state homes - Sudan Tribune_ Plural news and views on Sudan.txt
37_Hunger kills more than 40 people in S
Reading docs/37_Hunger kills more than 40 people in S.txt
38_Hunger looms in S
Reading docs/38_Hunger looms in S.txt
39_Millions could face famine in South Sudan, warns WFP - Sudan Tribune_ Plural news and views on Sudan
Reading docs/39_Millions could face famine in South Sudan, warns WFP - Sudan Tribune_ Plural news and views on Sudan.txt
40_sudantribune
Reading docs/40_sudantribune.txt
41_sudantribune
Reading docs/41_sudantribune.txt
42_extreme-weather-resilience-global-food-system
Reading docs/42_extreme-weather-resilience-global-food-system.txt
43_wfp280359
Reading docs/43_wfp280359.txt
44_wfp279459
Reading docs/44_wfp279459.txt
45_wfp278542
Reading docs/45_wfp278542.txt
46_wfp278541
Reading docs/46_wfp278541.txt
47_wfp287168
Reading docs/47_wfp287168.txt
48_wfp284277
Reading docs/48_wfp284277.txt
49_WFP-0000050282
Reading docs/49_WFP-0000050282.txt
50_wfp280481
Reading docs/50_wfp280481.txt
51_wfp289178
Reading docs/51_wfp289178.txt
52_i8533en
Reading docs/52_i8533en.txt
ERROR: [2018-07-30 09:57:14] indra/cwms - Could not parse XML string
ERROR: [2018-07-30 09:57:14] indra/cwms - Could not parse XML string
INFO: [2018-07-30 09:57:16] indra/hume - 911 relations of types causation, precondition, catalyst, mitigation, prevention found

We simply concatenate Statements from different sources - power of common knowledge representation!

In [123]:
statements = eidos_stmts + cwms_stmts + hume_stmts + sofia_stmts
print('%d total statements' % len(statements))
3172 total statements

Let's plot how many raw Statements we get from each source

In [124]:
source_cnt = sorted(Counter([st.evidence[0].source_api for st in statements]).items(),
                    key=lambda x: x[1], reverse=True)
plt.bar(*zip(*source_cnt))
Out[124]:
<BarContainer object of 4 artists>

Statements can be inspected manually / programmatically, and also serve as an exchange format

In [125]:
eidos_stmts[0]
Out[125]:
Influence(conflict(), livelihood(negative))
In [126]:
eidos_stmts[0].evidence[0].text
Out[126]:
'WHY  Persistent insecurity and armed conflict have disrupted livelihood activities , affected market functionality and limited physical access to markets .'
In [127]:
eidos_stmts[0].subj.db_refs['UN']
Out[127]:
[('UN/events/human/conflict', 0.8372545191211976),
 ('UN/events/human/intervention/intervention', 0.578830287706582),
 ('UN/entities/GPE', 0.5582304834451738),
 ('UN/entities/human/government/government_actions/regulation',
  0.5167213919557495),
 ('UN/events/crisis', 0.5124510124752975),
 ('UN/events/nature_impact/climate_change_mitigation', 0.5117896360536861),
 ('UN/events/human/human_migration', 0.5093050884962552),
 ('UN/entities/human/government/government_entity', 0.4985052191944197),
 ('UN/entities/human/government/government_actions/duty', 0.484892234234339),
 ('UN/entities/human/population', 0.48044607716068743)]
In [128]:
print(json.dumps(eidos_stmts[0].to_json(), indent=1))
{
 "type": "Influence",
 "subj": {
  "name": "conflict",
  "db_refs": {
   "TEXT": "armed conflict",
   "UN": [
    [
     "UN/events/human/conflict",
     0.8372545191211976
    ],
    [
     "UN/events/human/intervention/intervention",
     0.578830287706582
    ],
    [
     "UN/entities/GPE",
     0.5582304834451738
    ],
    [
     "UN/entities/human/government/government_actions/regulation",
     0.5167213919557495
    ],
    [
     "UN/events/crisis",
     0.5124510124752975
    ],
    [
     "UN/events/nature_impact/climate_change_mitigation",
     0.5117896360536861
    ],
    [
     "UN/events/human/human_migration",
     0.5093050884962552
    ],
    [
     "UN/entities/human/government/government_entity",
     0.4985052191944197
    ],
    [
     "UN/entities/human/government/government_actions/duty",
     0.484892234234339
    ],
    [
     "UN/entities/human/population",
     0.48044607716068743
    ]
   ],
   "WDI": [
    [
     "WDI/Presence_of_peace_keepers_(number_of_troops,_police,_and_military_observers_in_mandate)",
     0.7442294190168166
    ],
    [
     "WDI/Battle-related_deaths_(number_of_people)",
     0.7245324792078472
    ],
    [
     "WDI/Intentional_homicides_(per_100,000_people)",
     0.6978027975261973
    ],
    [
     "WDI/Legislation_exists_on_domestic_violence_(1=yes;_0=no)",
     0.6657021606771869
    ],
    [
     "WDI/Internally_displaced_persons,_new_displacement_associated_with_disasters_(number_of_cases)",
     0.6585000037437888
    ],
    [
     "WDI/Internally_displaced_persons,_new_displacement_associated_with_conflict_and_violence_(number_of_cases)",
     0.6585000037437888
    ],
    [
     "WDI/Internally_displaced_persons,_total_displaced_by_conflict_and_violence_(number_of_people)",
     0.6546640158765528
    ],
    [
     "WDI/Refugee_population_by_country_or_territory_of_origin",
     0.6174025676962379
    ],
    [
     "WDI/Refugee_population_by_country_or_territory_of_asylum",
     0.6056444643165692
    ],
    [
     "WDI/CPIA_property_rights_and_rule-based_governance_rating_(1=low_to_6=high)",
     0.6041915401228357
    ]
   ],
   "FAO": [
    [
     "FAO/events/Value/Prevalence of severe food insecurity in the total population",
     0.42605481572813997
    ],
    [
     "FAO/events/Value/Political stability and absence of violence\\/terrorism (index)",
     0.42574124730745916
    ],
    [
     "FAO/events/Gross Production Value (constant 2004-2006 1000 I$)/Meat, game",
     0.3769699666639242
    ],
    [
     "FAO/events/Gross Production Value (constant 2004-2006 1000 I$)/Honey, natural",
     0.3647201269248842
    ],
    [
     "FAO/events/Net Production Value (constant 2004-2006 1000 I$)/Meat, game",
     0.36062477278813493
    ],
    [
     "FAO/events/Value/Per capita food production variability (I$ per person constant 2004-06)",
     0.35226498211012797
    ],
    [
     "FAO/events/Value/Gross domestic product per capita, PPP (constant 2011 international $)",
     0.3517714280477182
    ],
    [
     "FAO/events/Net Production Value (constant 2004-2006 1000 I$)/Honey, natural",
     0.34849054100727217
    ],
    [
     "FAO/events/Implied emission factor for CH4 (Manure management)/Cattle, dairy",
     0.3428718326596775
    ],
    [
     "FAO/events/Implied emission factor for N2O (Manure management)/Cattle, dairy",
     0.34145595374827603
    ]
   ]
  }
 },
 "subj_delta": {
  "adjectives": [],
  "polarity": null
 },
 "obj": {
  "name": "livelihood",
  "db_refs": {
   "TEXT": "livelihood activities",
   "UN": [
    [
     "UN/entities/human/livelihood",
     0.6007075951618932
    ],
    [
     "UN/entities/human/farming",
     0.5792069073154111
    ],
    [
     "UN/entities/natural/crop_technology/management",
     0.5504081486065598
    ],
    [
     "UN/entities/human/financial/economic/revenue",
     0.5444352856229283
    ],
    [
     "UN/entities/human/financial/economic/poverty",
     0.5226406036753819
    ],
    [
     "UN/events/human/intervention/intervention",
     0.5194867760611138
    ],
    [
     "UN/entities/natural/biology/ecosystem",
     0.500102041234751
    ],
    [
     "UN/events/human/agriculture/farming",
     0.4995819055642296
    ],
    [
     "UN/events/nature_impact/forestry",
     0.4915727796187674
    ],
    [
     "UN/entities/natural/natural_resources/biotic_resources/biotic_resources",
     0.4858480054223513
    ]
   ],
   "WDI": [
    [
     "WDI/Informal_employment,_male_(%_of_total_non-agricultural_employment)",
     0.53967311648341
    ],
    [
     "WDI/Informal_employment,_female_(%_of_total_non-agricultural_employment)",
     0.53967311648341
    ],
    [
     "WDI/Informal_employment_(%_of_total_non-agricultural_employment)",
     0.53967311648341
    ],
    [
     "WDI/Total_fisheries_production_(metric_tons)",
     0.5193482280305918
    ],
    [
     "WDI/Contributing_family_workers,_total_(%_of_total_employment)_(modeled_ILO_estimate)",
     0.5179604764317277
    ],
    [
     "WDI/Contributing_family_workers,_male_(%_of_male_employment)_(modeled_ILO_estimate)",
     0.5179604764317277
    ],
    [
     "WDI/Contributing_family_workers,_female_(%_of_female_employment)_(modeled_ILO_estimate)",
     0.5179604764317277
    ],
    [
     "WDI/Self-employed,_male_(%_of_male_employment)_(modeled_ILO_estimate)",
     0.5090570416238172
    ],
    [
     "WDI/Self-employed,_female_(%_of_female_employment)_(modeled_ILO_estimate)",
     0.5090570416238172
    ],
    [
     "WDI/Self-employed,_total_(%_of_total_employment)_(modeled_ILO_estimate)",
     0.5090570416238172
    ]
   ],
   "FAO": [
    [
     "FAO/events/Density of livestock in the agricultural area/Equidae",
     0.45415396719859497
    ],
    [
     "FAO/events/Density of livestock in the agricultural area/Chickens",
     0.4190302709182986
    ],
    [
     "FAO/events/Indirect emissions (CO2eq) (Manure management)/Cattle, non-dairy",
     0.41138863010868637
    ],
    [
     "FAO/events/Density of livestock in the agricultural area/Major livestock types",
     0.41120117448868787
    ],
    [
     "FAO/events/Indirect emissions (CO2eq) (Manure management)/Cattle, dairy",
     0.41038674359778865
    ],
    [
     "FAO/events/Density of livestock in the agricultural area/Asses",
     0.40995012553917
    ],
    [
     "FAO/events/Direct emissions (CO2eq) (Manure management)/Cattle, non-dairy",
     0.4079390470859371
    ],
    [
     "FAO/events/Direct emissions (CO2eq) (Manure management)/Cattle, dairy",
     0.4045904585340253
    ],
    [
     "FAO/events/Indirect emissions (N2O) (Manure management)/Cattle, dairy",
     0.40238150409405565
    ],
    [
     "FAO/events/Indirect emissions (CO2eq) (Manure on pasture)/Cattle, dairy",
     0.3999574020947933
    ]
   ]
  }
 },
 "obj_delta": {
  "adjectives": [],
  "polarity": -1
 },
 "evidence": [
  {
   "source_api": "eidos",
   "pmid": "1_IPC_Alert_9_SouthSudan_Sept2017",
   "text": "WHY  Persistent insecurity and armed conflict have disrupted livelihood activities , affected market functionality and limited physical access to markets .",
   "annotations": {
    "found_by": "ported_syntax_1_verb-Causal",
    "provenance": [
     {
      "@type": "Provenance",
      "document": {
       "@id": "_:Document_1"
      },
      "documentCharInterval": [
       {
        "@type": "Interval",
        "start": 1940,
        "end": 1990
       }
      ],
      "sentence": {
       "@id": "_:Sentence_13"
      },
      "positions": [
       {
        "@type": "Interval",
        "start": 5,
        "end": 10
       }
      ]
     }
    ]
   }
  }
 ],
 "id": "b59ff86a-c473-4130-b36e-46ab5d4a3f68",
 "sbo": "http://identifiers.org/sbo/SBO:0000183"
}

Let's now start assembly for real! First, we map grounding between ontologies.

In [129]:
om = ontology_mapper.OntologyMapper(statements, ontology_mapper.wm_ontomap,
                                    symmetric=False, scored=True)
om.map_statements()
statements = om.statements
In [130]:
Image('demo/ontomap.png')
Out[130]:

Let's look at a specific example

In [131]:
statements[261].subj.db_refs
Out[131]:
{'CWMS': [('ONT::UNSAFE-SCALE|ONT::FOOD', 1.0)],
 'FAO': [('FAO/events/Value/Prevalence of severe food insecurity in the total population',
   0.7136202156818626),
  ('FAO/events/Value/Per capita food production variability (I$ per person constant 2004-06)',
   0.5980443219868409),
  ('FAO/events/Value/Number of severely food insecure people',
   0.5915045580547094),
  ('FAO/events/Value/Per capita food supply variability (kcal\\/capita\\/day)',
   0.5850729001861397),
  ('FAO/events/Yield\\/Carcass Weight/Meat, Poultry', 0.5745268362148006),
  ('FAO/events/Emissions intensity/Meat, chicken', 0.5718892276960331),
  ('FAO/events/Yield\\/Carcass Weight/Meat, cattle', 0.5619494705909741),
  ('FAO/events/Yield\\/Carcass Weight/Meat, chicken', 0.55498842249071),
  ('FAO/events/Food aid shipments/Non-Cereals', 0.549690556802688),
  ('FAO/events/Emissions intensity/Meat, cattle', 0.548090534021567)],
 'HUME': [('event/healthcare/famine', 0.7306410314328213)],
 'SOFIA': [('Health/Famine', 0.8113793459077407)],
 'TEXT': 'level of food insecurity this\nyear',
 'UN': [('UN/entities/human/food/food_insecurity', 0.7979051504774868),
  ('UN/entities/human/food/food_security', 0.7391947521416266),
  ('UN/entities/food_availability', 0.7349102160320637),
  ('UN/events/human/famine', 0.6420612664706928),
  ('UN/entities/natural/crop_technology/product', 0.640510191696878),
  ('UN/entities/human/financial/economic/poverty', 0.616514061032048),
  ('UN/entities/natural/crop_technology/management', 0.5941798020763639),
  ('UN/entities/human/livelihood', 0.58830886699566),
  ('UN/entities/natural/natural_resources/biotic_resources/biotic_resources',
   0.5491056345324559),
  ('UN/events/human/agriculture/food_production', 0.5465982851071531)],
 'WDI': [('WDI/Depth_of_the_food_deficit_(kilocalories_per_person_per_day)',
   0.6424394846324976),
  ('WDI/Prevalence_of_undernourishment_(%_of_population)', 0.6237492477216653),
  ('WDI/Urban_poverty_gap_at_national_poverty_lines_(%)', 0.6038757750823387),
  ('WDI/Rural_poverty_gap_at_national_poverty_lines_(%)', 0.6033984169387783),
  ('WDI/Food_production_index_(2004-2006_=_100)', 0.6006811488964685),
  ('WDI/Poverty_gap_at_national_poverty_lines_(%)', 0.5875257999303962),
  ('WDI/Poverty_gap_at_$5.50_a_day_(2011_PPP)_(%)', 0.5874904862205257),
  ('WDI/Poverty_gap_at_$3.20_a_day_(2011_PPP)_(%)', 0.5874904862205257),
  ('WDI/Urban_poverty_headcount_ratio_at_national_poverty_lines_(%_of_urban_population)',
   0.5867585486613438),
  ('WDI/Droughts,_floods,_extreme_temperatures_(%_of_population,_average_1990-2009)',
   0.5865961289979098)]}

We see here that the original string "food security" is grounded to the UN, HUME, SOFIA and CWMS ontologies simultaneously.

We tabulate each Concept by the ontologies it is grounded to. This shows that the majority of Concepts are now grounded to multiple ontologies, with 2500+ grounded to at least 3 ontologies.

In [132]:
cnt = Counter([tuple(sorted(list(set(conc.db_refs.keys())-{'TEXT', 'FAO', 'WDI'})))
                             for stmt in statements for conc in stmt.agent_list()])
df_from_counter(cnt, 'Grounding')
Out[132]:
Grounding count
0 (CWMS, HUME, SOFIA, UN) 675
1 (HUME, SOFIA, UN) 3793
2 (UN,) 654
3 () 26
4 (CWMS,) 1087
5 (CWMS, UN) 91
6 (HUME,) 18

Next, let's filter out Statements with unreliably grounded or ungrounded concepts

In [133]:
statements = ac.filter_grounded_only(statements, score_threshold=0.7)
INFO: [2018-07-30 10:00:20] indra/assemble_corpus - Filtering 3172 statements for grounded agents...
INFO: [2018-07-30 10:00:21] indra/assemble_corpus - 1786 statements after filter...

Now that we mapped ontologies, let's filter Statements by relevance with respect ot a list of UN terms.

In [134]:
Image('demo/prior_model.png', retina=True)
Out[134]:
In [135]:
statements = ac.filter_by_db_refs(statements, 'UN',
    ['conflict', 'food_security', 'food_insecurity', 'flooding', 'food_production',
     'human_migration', 'drought', 'food_availability', 'market',
     'precipitation'], policy='all',
    match_suffix=True)
INFO: [2018-07-30 10:01:11] indra/assemble_corpus - Filtering 1786 statements for those with all agents grounded to: conflict, food_security, food_insecurity, flooding, food_production, human_migration, drought, food_availability, market, precipitation in the UN namespace...
INFO: [2018-07-30 10:01:11] indra/assemble_corpus - 411 Statements after filter...

We make an assembly assumption in which we assume implicit positive polarity for influences where the subject polarity is not explicit. We then filter out any remaining Statements that don't have both subject and object polarity set.

In [136]:
assume_polarity(statements)
statements = filter_has_polarity(statements)
225 statements after polarity filter

Next, we run "preasembly" on the set of Statements. We first construct a joint hierarchy of all the reader ontologies.

In [137]:
def get_joint_hierarchies():
    eidos_ont = os.path.join(os.path.abspath(eidos.__path__[0]),
                             'eidos_ontology.rdf')
    trips_ont = os.path.join(os.path.abspath(cwms.__path__[0]),
                             'trips_ontology.rdf')
    hume_ont = os.path.join(os.path.abspath(hume.__path__[0]),
                           'hume_ontology.rdf')
    hm = HierarchyManager(eidos_ont, True, True)
    hm.extend_with(trips_ont)
    hm.extend_with(hume_ont)
    hierarchies = {'entity': hm}
    return hierarchies
hierarchies = get_joint_hierarchies()

The code below is usually a one-liner but here we break it up into multiple parts to show exactly what happens.

In [138]:
# Combine duplicates with the Preassembler
pa = Preassembler(hierarchies, statements)
unique_stmts = pa.combine_duplicates()
print('%d unique statements' % len(unique_stmts))
# Run the BeliefEngine with prior probabilities
be = BeliefEngine()
be.set_prior_probs(unique_stmts)
# Combine hierarchically-related Statements with the Preassembler
related_stmts = pa.combine_related(return_toplevel=False)
# Propagate beliefs over the hierarchy graph
be.set_hierarchy_probs(related_stmts)
INFO: [2018-07-30 10:02:39] indra/preassembler - Preassembling Influence (91)
91 unique statements

We can now filter for belief if we want to and get only the top-level Statements in the hierarchy

In [139]:
#statements = ac.filter_belief(related_stmts, 0.8)
top_stmts = ac.filter_top_level(related_stmts)
print('%d top-level statements' % len(top_stmts))
INFO: [2018-07-30 10:02:50] indra/assemble_corpus - Filtering 91 statements for top-level...
INFO: [2018-07-30 10:02:50] indra/assemble_corpus - 80 statements after filter...
80 top-level statements

↑ This set of Statements is the input to Demo 2

The sources of individual pieces of evidence that contribute to Statements overall:

In [140]:
render_stmt_graph(statements).draw('demo/stmt_graph.png', prog='dot')
In [141]:
cnt = Counter([ev.source_api for stmt in top_stmts for ev in stmt.evidence])
df_from_counter(cnt, 'Source')
Out[141]:
Source count
0 eidos 74
1 hume 93
2 sofia 15
3 cwms 4
In [142]:
cnt = Counter([tuple(sorted(list({ev.source_api for ev in stmt.evidence}))) for stmt in top_stmts])
df_from_counter(cnt, 'Sources')
Out[142]:
Sources count
0 (eidos,) 48
1 (eidos, hume) 5
2 (hume, sofia) 4
3 (hume,) 14
4 (cwms, eidos, hume) 1
5 (sofia,) 3
6 (eidos, sofia) 2
7 (eidos, hume, sofia) 1
8 (cwms, hume) 2

Let's pick one Statement and look at its evidence sentences in detail

In [155]:
stmt = [s for s in top_stmts if 'conflict' in s.subj.db_refs['UN'][0][0] and 'migration' in s.obj.db_refs['UN'][0][0]][0]
print('======')
print('source: %s' % stmt.subj.db_refs['UN'][0][0])
print('target: %s' % stmt.obj.db_refs['UN'][0][0])
print('======')
for ev in sorted(stmt.evidence, key=lambda x: x.text):
    print('%s: %s' % (ev.source_api.upper(), ev.text))
    print('--')
======
source: UN/events/human/conflict
target: UN/events/human/human_migration
======
SOFIA: Additionally , recent clashes have reportedly led to the displacement of at least 25 000 civilians around Ibba , Maridi , Mundri East , Mundri West and Yambio counties in Western Equatoria State . 
--
HUME: Although the  dispute  within  the  Sudan  People’s  Liberation  Movement  (SPLM)  was  primarily  political,  spiraling  violence  quickly  led  the  SPLM  and  the  national  army  (SPLA)  to  split,  resulting  in  the  formation  of  the  SPLM/A  in  Opposition  (SPLA-IO).
--
HUME: Conflict  is  driving  new  displacement,  putting  additional  stress  on  available  wild  food  sources.
--
SOFIA: Conflict continues to drive large-scale displacement , and the majority of those fleeing the country are leaving Greater Equatoria for Uganda ( Figure 4 ) . 
--
HUME: Fighting has also  been intense in the Greater Equatoria region, causing record  numbers of South Sudanese people to flee to Uganda, where the  number of refugees surpassed 1 million in 2017.
--
EIDOS: Further escalation of the conflict would likely result in increased food insecurity and displacement while humanitarian access and space may become further restricted .
--
EIDOS: IDPIDP POPULATION BYsite SITEtype TYPE population by in informal settlements  1.86  million internally displaced  with host communities Source : IOM/CCCM , Nov 2017 Source : IOM/CCCM , Nov 2017  18 18  Denotes a score of 0  Sources : CCCM , IOM/DTM  Sources : CCCM , IOM/DTM  During 2017 , new fighting and violence in Central Equatoria , Jonglei , Upper Nile and Western Bahr el Ghazal increased the population of IDPs in Wau by 20,000 and in Bor by 800 .
--
HUME: In  addition  to  the  life,  the  violence  has  caused  displacement,  restricted  households’  movement  towards  fields,  and  forced  aid  workers  to  relocate,  all  of  which  are  contributing to continued extreme acute food insecurity.
--
SOFIA: In the Greater Equatoria region , food security is likely to worsen significantly due to the ongoing conflict that has led to displacement of the local population which has disrupted the second season cropping . 
--
HUME: Likewise, conflict between Oromia and  Somali Regions in Ethiopia has also caused the displacement of hundreds of thousands of people in the two  regions.
--
HUME: Localized conflict  Escalation in localized conflict leading to continued displacement
--
HUME: Nyalen’s case is but one example  among millions of how conflict  causes hunger, destroys individual  and household resilience, and  undermines rural livelihoods.
--
HUME: The  conflict  is  driving  continued  large-scale  displacements,  particularly  of  farming  households  from  the  country’s greenbelt.
--
HUME: Thousands of people living with HIV have seen  their life-sustaining treatment interrupted without possibility  of resumption and fresh violence in Western and Central  Equatoria is affecting populations with the highest prevalence  rates of HIV/AIDS in South Sudan.
--
HUME: Violence and  displacement in the Greater Equatoria region have severely  affected populations with the highest prevalence of HIV/AIDS  and disrupted access to life-saving treatment.
--
HUME: these  include  continuing  economic  crisis  compounded  by  spiral  price  increases,  continued  inflation,  poor  harvests  and  early  depletion  of  household  stocks,  continued  population  displacements  resulting  from  localized  conflict,  trade  flows  hampered  by  insecurity,  disruption  of  livelihood  activities,  and  impeded  investments  and,  therefore,  revenue generation.
--
HUME: ï‚·  Armed  clashes  will  lead  to  further  internal  and  external  displacement.
--

We can also find contradictions

In [144]:
standardize_names(top_stmts)
pa.stmts = top_stmts
contradictions = pa.find_contradicts()
for c1, c2 in contradictions:
    print('%s\n <-> %s\n' % (c1, c2))
Influence(Food availability(negative), Food availability(negative))
 <-> Influence(Food availability(positive), Food availability(negative))

Influence(Food availability(negative), Food availability(negative))
 <-> Influence(Food availability(positive), Food availability(negative))

Influence(Food availability(positive), Food availability(negative))
 <-> Influence(Food availability(positive), Food availability(positive))

Influence(Food availability(positive), Food availability(negative))
 <-> Influence(Food availability(positive), Food availability(positive))

Influence(Food availability(positive), Food insecurity(negative))
 <-> Influence(Food availability(positive), Food security(negative))

Influence(Food availability(negative), Food security(negative))
 <-> Influence(Food availability(positive), Food security(negative))

Influence(Food availability(positive), Food security(negative))
 <-> Influence(Food availability(positive), Food security(positive))

Influence(Market(negative), Market(negative))
 <-> Influence(Market(positive), Market(negative))

Influence(Market(positive), Market(negative))
 <-> Influence(Market(positive), Market(positive))

Influence(Food insecurity(positive), Food availability(negative))
 <-> Influence(Food insecurity(positive), Food availability(positive))

Influence(Food insecurity(positive), Food availability(negative))
 <-> Influence(Food security(positive), Food availability(negative))

Influence(Food insecurity(positive), Food availability(positive))
 <-> Influence(Food security(negative), Food availability(negative))

Influence(Food insecurity(positive), Market(negative))
 <-> Influence(Food insecurity(positive), Market(positive))

Influence(Food insecurity(positive), Market(negative))
 <-> Influence(Food security(positive), Market(negative))

Influence(Food insecurity(positive), Market(positive))
 <-> Influence(Food security(positive), Market(positive))

Influence(Food insecurity(positive), Food insecurity(negative))
 <-> Influence(Food insecurity(positive), Food insecurity(positive))

Influence(Food insecurity(positive), Food insecurity(negative))
 <-> Influence(Food insecurity(positive), Food security(negative))

Influence(Food insecurity(positive), Food insecurity(negative))
 <-> Influence(Food security(negative), Food insecurity(positive))

Influence(Food insecurity(positive), Conflict(negative))
 <-> Influence(Food insecurity(positive), Conflict(positive))

Influence(Food insecurity(positive), Conflict(negative))
 <-> Influence(Food security(positive), Conflict(negative))

Influence(Food insecurity(positive), Conflict(positive))
 <-> Influence(Food security(positive), Conflict(positive))

Influence(Food insecurity(positive), Human migration(negative))
 <-> Influence(Food insecurity(positive), Human migration(positive))

Influence(Food insecurity(positive), Human migration(positive))
 <-> Influence(Food security(positive), Human migration(positive))

Influence(Food security(negative), Food availability(negative))
 <-> Influence(Food security(positive), Food availability(negative))

Influence(Food security(positive), Market(negative))
 <-> Influence(Food security(positive), Market(positive))

Influence(Food security(positive), Conflict(negative))
 <-> Influence(Food security(positive), Conflict(positive))

Influence(Conflict(positive), Food availability(negative))
 <-> Influence(Conflict(positive), Food availability(positive))

Influence(Conflict(positive), Market(negative))
 <-> Influence(Conflict(positive), Market(positive))

Influence(Conflict(positive), Food insecurity(negative))
 <-> Influence(Conflict(positive), Food insecurity(positive))

Influence(Conflict(positive), Food insecurity(negative))
 <-> Influence(Conflict(positive), Food security(negative))

Influence(Conflict(positive), Food insecurity(positive))
 <-> Influence(Conflict(positive), Food security(positive))

Influence(Conflict(positive), Food security(negative))
 <-> Influence(Conflict(positive), Food security(positive))

Influence(Conflict(positive), Human migration(negative))
 <-> Influence(Conflict(positive), Human migration(positive))

Influence(Human migration(positive), Food security(negative))
 <-> Influence(Human migration(positive), Food security(positive))

Influence(Human migration(positive), Food security(negative))
 <-> Influence(Human migration(positive), Food security(positive))

Influence(Flooding(positive), Market(negative))
 <-> Influence(Flooding(positive), Market(positive))

Influence(Flooding(positive), Food insecurity(negative))
 <-> Influence(Flooding(positive), Food insecurity(positive))

Influence(Precipitation(positive), Market(negative))
 <-> Influence(Precipitation(positive), Market(positive))

How can we decide between alternatives?

  1. Often an "apparent" conflict is resolved by additional temporal or spatial context
  2. We can decide based on relative belief
  3. We can expose these to the user an let them decide
  4. In some cases both can be left in the model and data can implicitly prioritize
  5. We can generate ensembles of models that can account for both possibilities (weighted by belief support)
In [145]:
top_stmts = remove_contradicts(top_stmts, contradictions)
INFO: [2018-07-30 10:05:52] indra/assemble_corpus - Filtering 80 statements for 38 UUIDs...
INFO: [2018-07-30 10:05:52] indra/assemble_corpus - 50 statements after filter...

Now let's move on to model assembly

In [146]:
model = assemble_pysb(top_stmts)

This model works and can be simulated but it's not parameterized! Demo 3 covered methods to automatically parameterize models via gradable adjectives and indicators.

In [147]:
res_df = simulate_model(model)
In [148]:
res_df[['Conflict', 'Food_insecurity', 'Food_production']]
Out[148]:
Conflict Food_insecurity Food_production
time
0.000000 10000.000000 10000.000000 10000.000000
10.101010 8223.970178 10052.925638 10012.497243
20.202020 6770.038587 10103.014174 10025.650537
30.303030 5579.780744 10150.787503 10039.335379
40.404040 4605.422011 10196.672039 10053.450015
50.505051 3807.816621 10241.016932 10067.911098
60.606061 3154.942768 10284.107599 10082.650457
70.707071 2620.562228 10326.177768 10097.612232
80.808081 2183.183913 10367.418769 10112.750662
90.909091 1825.231124 10407.986992 10128.028313
101.010101 1532.310954 10448.010427 10143.414517
111.111111 1292.629073 10487.593894 10158.884131
121.212121 1096.531955 10526.823132 10174.416562
131.313131 936.119121 10565.768264 10189.994946
141.414141 804.922197 10604.486655 10205.605464
151.515152 697.643933 10643.025248 10221.236785
161.616162 609.947675 10681.422433 10236.879625
171.717172 538.283938 10719.709596 10252.526374
181.818182 479.745825 10757.912407 10268.170788
191.919192 431.953416 10796.051850 10283.807749
202.020202 392.958696 10834.145066 10299.433055
212.121212 361.166491 10872.206062 10315.043260
222.222222 335.270875 10910.246273 10330.635532
232.323232 314.202781 10948.275028 10346.207546
242.424242 297.086849 10986.299937 10361.757390
252.525253 283.206514 11024.327199 10377.283489
262.626263 271.975099 11062.361857 10392.784550
272.727273 262.912267 11100.408013 10408.259503
282.828283 255.624846 11138.468994 10423.707467
292.929293 249.790891 11176.547493 10439.127714
... ... ... ...
707.070707 250.067209 12759.815554 11044.864388
717.171717 250.796134 12798.965706 11058.970860
727.272727 251.525377 12838.139215 11073.045007
737.373737 252.254885 12877.335915 11087.086799
747.474747 252.984663 12916.555639 11101.096208
757.575758 253.714658 12955.798222 11115.073202
767.676768 254.444823 12995.063497 11129.017754
777.777778 255.175088 13034.351298 11142.929834
787.878788 255.905422 13073.661458 11156.809413
797.979798 256.635882 13112.993810 11170.656463
808.080808 257.366387 13152.348187 11184.470956
818.181818 258.096920 13191.724423 11198.252864
828.282828 258.827504 13231.122349 11212.002159
838.383838 259.558134 13270.541799 11225.718814
848.484848 260.288820 13309.982605 11239.402801
858.585859 261.019533 13349.444599 11253.054093
868.686869 261.750296 13388.927614 11266.672663
878.787879 262.481116 13428.431481 11280.258485
888.888889 263.211969 13467.956034 11293.811532
898.989899 263.942856 13507.501104 11307.331778
909.090909 264.673762 13547.066524 11320.819197
919.191919 265.404697 13586.652124 11334.273763
929.292929 266.135642 13626.257738 11347.695452
939.393939 266.866581 13665.883197 11361.084237
949.494949 267.597524 13705.528332 11374.440093
959.595960 268.328463 13745.192976 11387.762996
969.696970 269.059395 13784.876959 11401.052921
979.797980 269.790318 13824.580114 11414.309844
989.898990 270.521232 13864.302272 11427.533740
1000.000000 271.252142 13904.043263 11440.724587

100 rows × 3 columns

We can also use INDRA's explanation module and the ModelChecker to find paths that satisfy a given set of overall Influences (with polarities).

In [149]:
from indra.explanation.model_checker import ModelChecker
to_check = Influence(Concept('Conflict'), Concept('Food_insecurity'))
mc = ModelChecker(model, statements=[to_check])
mc.prune_influence_map()
paths = mc.check_model(max_paths=10)
paths[0][1]
INFO: [2018-07-30 10:06:43] indra/pysb_assembler - No monomer found corresponding to agent Food_insecurity
INFO: [2018-07-30 10:06:43] indra/model_checker - Generating influence map
INFO: [2018-07-30 10:06:43] indra/model_checker - Removing self loops
INFO: [2018-07-30 10:06:43] indra/model_checker - Removing self loop: ('Market_degrades_Market', 'Market_degrades_Market')
INFO: [2018-07-30 10:06:43] indra/model_checker - Removing self loop: ('Food_production_degrades_Food_production', 'Food_production_degrades_Food_production')
INFO: [2018-07-30 10:06:43] indra/model_checker - Removing self loop: ('Precipitation_degrades_Precipitation', 'Precipitation_degrades_Precipitation')
INFO: [2018-07-30 10:06:43] indra/model_checker - Get successorts of each node
INFO: [2018-07-30 10:06:43] indra/model_checker - Compare combinations of successors
INFO: [2018-07-30 10:06:43] indra/model_checker - Removing 44 edges from influence map
INFO: [2018-07-30 10:06:43] indra/model_checker - Checking stmt: Influence(Conflict(), Food_insecurity())
INFO: [2018-07-30 10:06:43] indra/model_checker - Running path finding with max_paths=10, max_path_length=5
INFO: [2018-07-30 10:06:43] indra/model_checker - Finding paths between Conflict() and Food_insecurity__obs with polarity 1
INFO: [2018-07-30 10:06:43] indra/model_checker - Finding paths between Conflict() and Food_insecurity__obs with polarity 1
Out[149]:
PathResult:
    path_found: True
    result_code: PATHS_FOUND
    path_metrics: 
        0: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 1
        1: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 2
        2: source_node: Conflict_degrades_Food_availability, target_node: Food_insecurity__obs, polarity: 1, length: 2
        3: source_node: Conflict_degrades_Food_production, target_node: Food_insecurity__obs, polarity: 1, length: 2
        4: source_node: Conflict_produces_Human_migration, target_node: Food_insecurity__obs, polarity: 1, length: 3
        5: source_node: Conflict_degrades_Food_availability, target_node: Food_insecurity__obs, polarity: 1, length: 3
        6: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 3
        7: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 3
        8: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 3
        9: source_node: Conflict_produces_Human_migration, target_node: Food_insecurity__obs, polarity: 1, length: 3
        10: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 3
        11: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        12: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        13: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        14: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        15: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        16: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        17: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        18: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        19: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        20: source_node: Conflict_degrades_Food_availability, target_node: Food_insecurity__obs, polarity: 1, length: 4
        21: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        22: source_node: Conflict_produces_Human_migration, target_node: Food_insecurity__obs, polarity: 1, length: 4
        23: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        24: source_node: Conflict_produces_Food_insecurity, target_node: Food_insecurity__obs, polarity: 1, length: 4
        25: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 4
        26: source_node: Conflict_produces_Market, target_node: Food_insecurity__obs, polarity: 1, length: 5
        27: source_node: Conflict_produces_Human_migration, target_node: Food_insecurity__obs, polarity: 1, length: 5
    paths: 
        0: ('Conflict_produces_Food_insecurity', 1)
           ('Food_insecurity__obs', 1)
        1: ('Conflict_produces_Market', 1)
           ('Market_produces_Food_insecurity', 1)
           ('Food_insecurity__obs', 1)
        2: ('Conflict_degrades_Food_availability', 1)
           ('Food_availability_degrades_Food_insecurity', -1)
           ('Food_insecurity__obs', 1)
        3: ('Conflict_degrades_Food_production', 1)
           ('Food_production_degrades_Food_insecurity', -1)
           ('Food_insecurity__obs', 1)
        4: ('Conflict_produces_Human_migration', 1)
           ('Human_migration_produces_Market', 1)
           ('Market_produces_Food_insecurity', 1)
           ('Food_insecurity__obs', 1)
        5: ('Conflict_degrades_Food_availability', 1)
           ('Food_availability_degrades_Market', -1)
           ('Market_produces_Food_insecurity', 1)
           ('Food_insecurity__obs', 1)
        6: ('Conflict_produces_Market', 1)
           ('Market_produces_Conflict', 1)
           ('Conflict_produces_Food_insecurity', 1)
           ('Food_insecurity__obs', 1)
        7: ('Conflict_produces_Food_insecurity', 1)
           ('Food_insecurity_degrades_Food_availability', 1)
           ('Food_availability_degrades_Food_insecurity', -1)
           ('Food_insecurity__obs', 1)
        8: ('Conflict_produces_Human_migration', 1)
           ('Human_migration_degrades_Food_availability', 1)
           ('Food_availability_degrades_Food_insecurity', -1)
           ('Food_insecurity__obs', 1)
        9: ('Conflict_produces_Food_insecurity', 1)
           ('Food_insecurity_degrades_Food_production', 1)
           ('Food_production_degrades_Food_insecurity', -1)
           ('Food_insecurity__obs', 1)
    max_paths: 10
    max_path_length: 5
In [102]:
paths[0][1].paths[8]
Out[102]:
(('Conflict_produces_Human_migration', 1),
 ('Human_migration_degrades_Food_availability', 1),
 ('Food_availability_degrades_Food_insecurity', -1),
 ('Food_insecurity__obs', 1))

↑ this is a powerful explanation finding tool:

"in what ways could increased precipitation have resulted in reduced food security?"

but for now, we can use it for debugging, i.e. "do the ways in which Precipitation affects Food insecurity in the model make sense?". In a lot of the cases, clearly, no, and these are easily identifiable in a problem-driven way here (i.e. focus on problems in the model that matter).

Finally, let's look at an integrated simulation of the model with Topoflow in a relevant context.

In [150]:
Image('demo/emeli_integ.png')
Out[150]:
In [151]:
ac.dump_statements(top_stmts, 'indra_eval_stmts.pkl')
INFO: [2018-07-30 10:08:31] indra/assemble_corpus - Dumping 50 statements into indra_eval_stmts.pkl...
In [152]:
from indra.assemblers.bmi_wrapper import BMIModel
model.name = 'indra_eval_model'
bmi_model = BMIModel(model)
out_name_maps = {'atmosphere_water__rainfall_volume_flux':            
                 'Precipitation'}                                     
input_vars = ['Precipitation']
bm = BMIModel(model, inputs=input_vars, stop_time=10000,           
              outside_name_map=out_name_maps)
bmi_model.export_into_python()

Topoflow is configured with precipitation for the Gel-Aliab basin

In [153]:
Image('demo/rivers.jpg', retina=True)
Out[153]:

We now use this configuration of Topoflow and simulate it together with the model we just assembled.

In [154]:
Image('demo/topoflow_simul2.png')
Out[154]: